Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fixed auto update from airflow variables #226

Merged
merged 25 commits into from
Oct 27, 2023
Merged

fixed auto update from airflow variables #226

merged 25 commits into from
Oct 27, 2023
+35 −3

Conversation

cayod
Copy link
Contributor

@cayod cayod commented Oct 6, 2023

This PR fixes the airflow variables update that wasn't happening in ci and release

@cayod cayod requested a review from a team as a code owner October 6, 2023 10:16
@cayod
Copy link
Contributor Author

cayod commented Oct 9, 2023

For the new changes to work in ci and release, we need to grant the service account that deploys the dags the following permissions:
composer.environments.create
composer.environments.executeAirflowCommand
composer.environments.update

The service accounts are:
[email protected]
[email protected]

@sydneynotthecity
Copy link
Contributor

@cayod both service accounts have been granted the listed permissions. Please try the deploy-to-dev job again to validate.

@cayod
Copy link
Contributor Author

cayod commented Oct 11, 2023

The CI failed, I was wrong about this permission. Can we grant Environment and Storage Object User role to this service account? @sydneynotthecity

@sydneynotthecity
Copy link
Contributor

@cayod I updated to grant permissions to the appropriate buckets. Can you test again?

@cayod
Copy link
Contributor Author

cayod commented Oct 16, 2023 •
edited
Loading

Sorry, I was wrong about this last role. Can we grant the Composer Environment Admin role to this service account? This is the right role for uploading the variables in composer. @sydneynotthecity

@sydneynotthecity
Copy link
Contributor

@cayod updated the permissions again

sydneynotthecity
sydneynotthecity reviewed Oct 26, 2023
View reviewed changes
Copy link
Contributor

@sydneynotthecity sydneynotthecity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of questions to make sure the release behavior is the same in production as test.

.github/workflows/release.yml Show resolved Hide resolved
.github/workflows/release.yml Outdated Show resolved Hide resolved
dags/stellar_etl_airflow/add_files_to_composer.py Outdated Show resolved Hide resolved
dags/stellar_etl_airflow/add_files_to_composer.py Show resolved Hide resolved
@sydneynotthecity sydneynotthecity merged commit 4bc95a0 into master Oct 27, 2023
4 checks passed
@sydneynotthecity sydneynotthecity deleted the auto-update-variables-airflow branch October 27, 2023 16:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sydneynotthecity sydneynotthecity sydneynotthecity approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@cayod @sydneynotthecity